Martingale problems and linear programs for singular control ∗

Using a controlled Jackson network as the primary example, stochastic control problems are formulated as martingale problems and the optimal solutions are characterized as the solutions of infinite dimensional linear programs. Assuming a heavy traffic scaling, the analogous linear program for a singular control problem is derived, and the solution of the linear program is shown to give the stochastic model minimizing the long-run average cost. 1 The model. We consider a controlled Jackson network with m stations, external arrival rates v1, . . . , vm service rates u1, . . . , um, and routing probabilities pij. We assume that the routing probabilities are fixed but that we can control the arrival and service rates subject, perhaps, to certain constraints. We can formulate the model as the solution of a system of stochastic equations: Q(t) = Q(0) + Yi( ∫ t